The present invention relates to monolithically integrated image processing systems and more in particular to an adaptive optical sensor, monolithically integrated together with control and image processing circuitry.
Integrated optical sensors are becoming very useful in a number of application fields. Fundamentally, the sensor constitutes the image detector (video camera) in a certain format, for example in the QCIF format (176.times.144 pixels) and advantageously is monolithically integrated with the processing and control circuit. Integrated sensors of this type are useful in industry for realizing cameras suitable for functioning in hostile environments, for alarm and monitoring systems, for implementing of intelligent motion sensors and for filming in multimedia environments, PC, low OCR resolution, FAX and the like.
The following publications treat various aspects of the state of the art technology used for fabricating this type of optical sensors, all of which are hereby incorporated by reference.
"An Object Position and Orientation IC with Embedded Imager", David L. Standley, JSSC Vol. 26, n. Dec. 12, 1991; PA1 "Smart Sensor Interface with A/D Conversion and Programmable Calibration", P. Malcovati, C. Azzeredo Leme, P. O'Leary, F. Maloberti and H. Baltes, JSSC Vol. 29, n. Aug. 8, 1994; PA1 "A New MOS Imager Using Photodiode as Current Source", Mikio Kyomasu, JSSC Vol. 26, n. Aug. 8, 1991; PA1 "Image Motion Detection Using Analog VLSI", Chu Phoon Chong, C. Andre, T. Salama and K. C. Smith, JSSC Vol. 27, n. Jan. 1, 1992, PA1 "Smart Pixel Cellular Neural Network in Analog Current Mode CMOS Technology", S. Espejo et al., JSSC Vol. 29, n. Aug. 8, 1994; PA1 "A 256.times.256 CMOS Active Pixel Image Sensor with Motion Detection", A. Dickinson, et al., ISSCC ISSN 0193-6530, February 1995, San Francisco.
The photosensitive portion of the integrated device is commonly constituted by an array or matrix of elementary cells, each corresponding to a pixel of a picked-up image (frame). The photodetector member of each cell may be a phototransistor or a photodiode. The luminous intensity of each pixel of an image optically projected over the area occupied by the array of photosensitive cells can be detected in terms of the current photogenerated by the photodetecting component of the respective cell.
The reading of an image detected by the array of photosensitive cells takes place by scanning the cells in cyclic succession, usually row by row. For each row of selected cells, the values relative to the single pixels of the row are commonly read in a parallel mode by a number of reading circuits equal to the number of pixels per row (i.e. equivalent to the number of columns of the photosensitive array). The values, read simultaneously for all the pixels of a selected row are "discharged" in a parallel mode, for example through a parallel-to-serial converter, in order to produce as output, a serial video signal including appropriate row synchronism pulses.
An alternative to the use of phototransistors or photodiodes is photomodulated sources of a current proportional to the luminous intensity, which, adequately amplified, provides a pixel signal that may be sampled in the reading phase, stored and/or processed in various ways according to particular algorithms. The common use of a photodiode and of an associated capacitance that may be switched in parallel to the diode for a preestablished period of time for storing the value of luminous intensity of the respective pixel in terms of electric charge, has the advantage of allowing for a great simplification of the integrated structure of the individual cell.
Transfer of the charge photogenerated by the photodiode to the respective storing capacitance can be controlled by a dedicated integrated switch.
Normally, the subsequent reading destroys the information (pixel) temporarily stored in the cell capacitance, thus resetting the state of charge of the storing capacitance to a preset level. Therefore, the cell is ready again to record the next frame.
The reading of the state of charge of the capacitance is commonly enabled by a dedicated integrated read select switch, capable of coupling the storing capacitance of the cell (pixel) to the input of a charge amplifier, common for all the cells of a column of the cell array (that is, for pixels of the same order of all the rows).
A typical cell arrangement in rows and columns is shown in FIG. 1. The figure schematically illustrates the read select switches Rs and the array of the reading charge amplifiers Ra.
A frame is read selecting one row at the time in succession. The pixel values, produced at the outputs of the charge amplifiers Ra, for each selected row, are delivered in parallel and converted into a serial signal by a dedicated parallel-to-serial converter circuit (not shown in the figure).
These systems require means for preventing saturation of the photosensitive elements (cells) and optimizing, that is adapting the sensitivity of the sensor to the illumination conditions of the scene or of the filmed subject. This requisite is of paramount importance because the variation of illumination between successive frames has a great impact on the global performances algorithm of analysis, principally based on the luminance component of the frames, especially so in the case of black and white videocameras.
Commonly, this function is electronically implemented through normal automatic gain control (AGC) loops, in function of luminance. A luminance signal can be generated in various ways. Often it is obtained from a dedicated sensor (exposure meter), integrated on the same device. In other systems, an average value of the frame luminance is obtained by processing the video signal itself. As a whole, the known systems rely on controlling the gain of a video signal amplifier, "downstream" of the sensor, through automatic gain regulation techniques.
These sensitivity regulation systems are intrinsically complex. There is a need and/or utility of an "adaptive" sensor, capable of automatically implementing a control of its own sensitivity in function of the illumination level of the filmed subject, that is of the luminance of each frame without requiring the realization of burdensome signal processing circuits.
This aim is fully attained by the present invention whose object is a method and relative sensor's architecture, capable of exerting a sensitivity control of the optical sensor in response to the level of illumination of the subject or of the frame's luminance.
Therefore, control is implemented at each photogram (frame) by the sensor in an essentially adaptive way.
In a sensor operating in a cyclic mode that stores each frame in the form of electric charge, a preferred method of the invention consists of detecting, during a first phase of each cycle, the value of the global electric current resulting from the sum of the currents photogenerated by all the photosensitive elements that make up the sensor and in regulating the (shutter) closing interval of the switches through which the photogenerated current is integrated in the storage capacitance of each photosensitive cell, substantially in accordance with an inverse proportionality law in function of the preliminary assessed level of the global current.
The successive "reading" of the charge level of the storage capacitance, that corresponds to the relative intensity value of each pixel of the frame, eventually produces a video signal whose amplitude has already been adjusted on the ground of the frame's luminance level, without the need of further signal processing and compensation of the video signal.
According to an important aspect of the invention, this sensor adaptivity can be implemented with a particularly simple cell structure, realizable in a completely compatible way by a standard CMOS fabrication process, as it shown hereafter.
The processing circuitry of the global current and that for generating the appropriate control signals that establish the interval of integration of the current photogenerated in each photosensitive element, typically a photodiode, in its respective storage capacitance, in function of the preliminary measured global photogenerated current, can be realized in an essentially analog form or in an almost entirely digital form. Alternatively, its function can be partly implemented by processing via software an information relative to the global current in order to generate, via software, the desired time interval during which the frame's pixel values are stored as electric charge.
According to a fundamental aspect of the invention each individual cell comprises, in integrated form, beside its photosensitive structure, for example a photodiode, a storage (MOS) capacitance, a first switch for coupling the storage capacitance in parallel to the photodiode and a second read-select switch (discharge) for discharging the electric charge stored in the capacitance through the input of the charge amplifier (reading amplifier), and also a third integrated switch. The latter has the function of connecting in parallel all the sensor photodiodes to a common line during an initial phase of each cycle. During this first phase of each cycle, the global current photogenerated by all the sensor cells is coupled, through a low impedance coupling circuit, to the input of a resettable charge amplifier. Therefore, the voltage ramp of the output node of the charge amplifier can be compared with a reference voltage by a comparator.
The interval of time between the initial instant of a new cycle and the triggering of the comparator provides the primary information for enabling, based on other prearrangeable parameters, the closing time of the switch for integrating the photogenerated current of the photodiode its respective storage capacitance, thus implementing the desired adaptive function of the sensor to the changing illumination conditions, on a frame-by-frame basis. In practice, the system of the invention implements an automatic regulation of a parameter that can be equated to the exposure time (i.e. the shutter of a classical camera system). In other words, the invention realizes an electronic shutter whose opening time is automatically regulated in terms of illumination condition of the subject, on a frame-to-frame basis.